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(57) Abstract 

Displaying a facsimile message with bit-mapped text on a 
small screen computer such as a personal digital assistant is difficult 
because of the limited screen size. This invention teaches a scrollable 
text selection window extending the full width of a miniaturised and 
therefore illegible image of the received facsimile; the height of the 
text selection window is such that only a single line of bit-mapped 
text is covered by it. The text selection window is divided into 
four segments of equal width; the bit-mapped text in the far left 
segment is magnified to legibility and then displayed as a first row in 
a viewing window. The next segment, moving rightwards over the 
line of text, is also subject to magnification and placed in a row of 
the viewing window below the highest row. The process is repeated 
for all four segments; the original line is therefore broken into four 
sections which are displayed as four rows in the viewing window 
enabling the line to be easily read in the viewing window. 
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Method of and apparatus for displaying an image of a document on a display 

5 Field of the Invention 

This invention relates to a method of and apparatus for displaying an image of a 
document on a display. In particular, it relates to a method of and apparatus for 
displaying an image of a bit-mapped text document on a display, in which the text 
document has been received from a physically remote transmitting device, such as a 
10 remote facsimile machine. 

Description of the Prior Art 

Many computers and personal communicators can be used with fax-modems that enable 
facsimile or other messages to be received. However, the small screen size and 

15 resolution of a typical display screen, particularly a portable computer or personal 
communicator display screen, can make reading messages difficult. For example, take a 
facsimile message sent from a conventional facsimile machine to a portable computer: 
The original message would be on A4 size paper or similar. The screen size of the 
portable computer is far less than A4, so that it is impossible to display the entire 

20 message at its original size. The problem can be put as one of being unable to see the 
necessary detail if one sees any significant part of the whole. 

One conventional approach is to display a portion of the bit-mapped image at a reduced 
magnification, with the user being able to select a region of the displayed bit-mapped 
25 image for extra magnification. Often, the user can call up a window that acts as if it is a 
"magnifying glass", magnifying the text underlying the window so that it can be read. 
In this way, a user can read words falling within a typically rectangular frame and can 
scroll the magnifying glass over the displayed portion of the text so that the text can be 
viewed. One disadvantage with this approach is that it is inconvenient to the user since 
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the user must carefully control the scrolling of the magnifying glass to read even a 
single line of text. Returning to the start of the next line is often difficult. This is 
particularly the case for computers or telephones with small screens, such as personal 
digital assistants or mobile communicators. 

5 

Another conventional approach is to attempt to recognise text represented by bit- 
mapped images using character recognition software. Once the bit-mapped text is 
converted to data, then it is easy to display the entire message by enabling the user to 
scroll through it. However, this approach generally loses most or all of the format of 
10 the original image, which may be inconvenient. More significantly, character 
recognition software is not sufficiently accurate for this task, is expensive, slow in 
operation and requires significant processing capabilities that may be beyond simpler 
personal digital assistants. 



15 Further reference may be made to GB 2298102 A to Samsung Aerospace Industries 
Ltd, This specification discloses an overhead projector apparatus. It does not however 
deal with the problem of displaying a bit-mapped text document on a display, in which 
the text document has been received from a physically remote transmitting device. 

20 Statement of the Invention 

In accordance with the present invention, a method of displaying an image of a 
document on a display screen comprises the steps of: 

[1] receiving, from a remote transmitting device, a bit-mapped image of a 
document comprising text; 
25 [2] dividing a whole or part of a line of text of the document into segments; 

[3] processing the segments so that text in each segment can be displayed at 
a legible size and re-arranging segments which are positioned adjacently 
in a line or a part of a line into two or more rows and 
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[4] displaying the re-arranged segments on the display screen. 

Using this approach, a bit mapped image of text, such as a facsimile image, can be 
displayed on a computer screen so that the text can be easily read. 

For example, a single line of bit-mapped text could be divided into four segments of 
equal width, with the bit-mapped text from each segment being processed to a legible 
size and then displayed within a window divided into four rows, with the first row 
containing the processed text from the segment from the far left of the unmagnified line, 
the next row down containing the processed text from the next segment along and so 
forth. 

It is not a requirement of this invention that the re-arrangement be done after the 
processing step: it is entirely equivalent to re-arrange and then process. 

Preferably, the following step occurs after step [2]: displaying either some or all of the 
image either (i) at a magnification at which it is not readily legible or (ii) as a picture 
representative of the image. This approach allows the user to select the data which is to 
be extracted and processed from the document image. For example, where that data 
corresponds to an entire line of bit-mapped text, then the user can select which 
particular line is to be extracted and re-assembled in magnified form. This can be 
achieved by providing a scrollable window (a ^'text selection window") or marker 
which can be moved to overlie the low magnification, illegible, portion of the image to 
be re-assembled. Alternatively, the low magnification, illegible image can be scrolled 
past a fixed marker such as a window covering the entire width of the image, which 
overlies that portion of the image to be re-assembled in magnified form. 
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It is convenient if only a single line of text is processed for re-assembly in magnified 
form* However, the hieight of characters, the line spacing etc. varies from document to 
document and may vary even within a document itself. It is therefore not possible to 
provide a fixed height narrow, rectangular text selection window [or other marker] for 
5 example, which will work with all documents. This problem can be solved by allowing 
the user to vary the height of such a window [or other marker]. For example, on the 
screen, the text selection window can be defined by a single line marking the centre of 
the short axis of the window; on screen buttons can then be provided which when 
selected either increase or decrease the height of the window. Another approach is to 
10 provide "grab handles" at some or all corners of the window; these can be dragged to 
enlarge or decrease the height of the window. 

Yet another alternative is to provide pattem recognition which can determine the height 
of a line of text, by for example, looking at a pre-determined narrow region of the text 
15 selection window and decreasing the height of that window in incremental steps until an 
output from a pattem recognition engine indicates that it most closely matches the 
pattern associated with a single line of text, preferably with a small amount of unfilled 
space above and below the line of text. 

20 Alternatively, the window may be dispensed with entirely: the system automatically 
work out work out the positions of each line of text using a pattem recognition engine. 
This removes the need for manual selection via a text selection window. Either 
approach can be implemented by those skilled in the recognisor arts. Further, since 
many of the devices for which this invention will be most apt will anyway incorporate 

25 hand-writing recognition software, the extra burden this approach places on cost and 
performance is minimal. 
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Some areas of the image may not be full width lines of text. Conveniently, the width of 
the window [or other marker] can be controlled by the user, for instance by dragging 
"grab handles" at the comers of the window until the appropriate width is reached* 

5 Also, the window [or other marker] can be used to indicate, at the user's option, either 
a conventional "magnifying glass" region or the re-assembly and magnification 
approach of this invention. For example, if the image comprises both text and some 
pictures, then the user may wish to view the text using the method of the present 
invention, but switch to the conventional "magnifying glass" approach to view the 
10 picture. In either option, the magnified image portions may be arranged to appear 
within the same window. Further, on-screen control buttons can present the user with 
the option of selecting either mode. 

In some devices, it may be impractical or undesirable to display the actual image, even 
15 in unmagnified form. This may be because, with the resolution available, the 
appearance of such an image is unattractive. Then, a picture merely representative of the 
image may be displayed and it would be against this picture that the user would place 
the text selection window or marker that indicated which portions were to be magnified. 
The picture would preferably have some detailing corresponding to the layout of the 
20 text in the image, although this is not essential. 

In another aspect, an apparatus for displaying an image of a document on a screen 
comprises: 

[1] a receiver for receiving, from a remote transmitting device, a bit-mapped 
25 image of a document comprising text; 

[2] a processor programmed (i) to process the document by dividing a 
whole or part of a line of text into segments; (ii) to process the segments so that text in 
each segment can be displayed at a legible size and (iii) to re-arrange the segments 
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which are positioned adjacently in a line or a part of a line into two or more rows so that 
their contents can be easily read and 

[3] a display to display the re-arranged segments. 

5 

Brief Description of the Drawings 

An embodiment of the invention will now be further described with reference to the 
accompanying drawings in which:- 

10 Figure 1 is a schematic block diagram illustrating the major components of such 

an apparatus for displaying an image of a document on a screen in accordance 
with the present invention; 

Figure 2 is a plan view of such an apparatus showing what might be displayed 
by such an apparatus, 

15 



Detailed Description 

The invention will now be described with reference to a particular embodiment. It will 
20 be appreciated that this embodiment includes various features that are not essential to 
each and every implementation of the invention. 

Referring now to Figure 1, a display means 2, comprising a LCD type display, is 
shown forming part of a computer indicated generally at 1 . The computer comprises a 
25 translucent transducer pad 3 overlaying the display 2, together with a stylus 4, The 
pad 3 is an Indium Tin Oxide (TTO) device which is well known in this art. It is 
operable to supply a signal indicative of the position at which the stylus 4 touches the 
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pad 3 and allows the user to easily and naturally input graphical information for storage 
and subsequent, although nearly immediate, display by the apparatus. 

It will be appreciated that a variety of components, e.g. a power supply, is necessary 
5 for the functioning of the apparatus and will have to be present in the apparatus but 
these are not expressly detailed here in the interests of brevity and clarity. 

The transducer ITO pad 3, also known as a digitiser, is shown schematically overlaying 
the LCD type display 2. The stylus 4 is touching the pad 3 and electrical signals 
10 indicative of die position (X Y co-ordinates) at which it contacts the pad 3 are supplied 
from the stylus 4 to an A to D converter 14 feeding a data path or system bus 10, 

A CPU 12 is connected for bi-directional data flow to the bus 10, as is a RAM 17. A 
ROM 18 is also connected to bus 10, being programmed with the operating system for 

15 the apparatus. CPU 12 provides all control signals via bus 10. RAM 17 stores, inter 
alia, the data defining received bit-mapped facsimile messages. RAM 17 may be an 
EEPROM or other form of memory. CPU 12 may be a microprocessor such as an 
ARM RISC chip. In addition there is provided a serial data interface 19 connected to 
bus 10 for sending and receiving information. A display driver 1 1 is connected to bus 

20 10 and drives the display 2. 

In use, when the user receives a facsimile, data defining the bit-mapped facsimile image 
is sent via the serial interface 19 from a fax/modem [not shown] and is stored in RAM 
17 or other storage medium, A reduced size version of the image, with much 
25 information lost, is displayed on the display 2 at 21. The text in facsimile image 21 is 
not legible. To read the text, the user selects the "FaxView" [TM] function from a drop 
down menu or other conventional system. 
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When Fax View is selected, text selection window 22 appears on screen over a portion 
of the facsimile image 21. The user can move text selection window 22 around 
facsimile image 21 using the conventional 4-way movement controller 23. In addition, 
in a pen driven system, the user can simply drag the text selection window 22 to the 
5 desired location. The height of the text selection window 22 may be automatically 
varied to match the height of the text in the facsimile by a pattern recognition engine as 
described earlier, in those implementations where a recognisor engine is available. The 
details of the pattern recognition engine will be familiar to the skilled implementer. In 
addition, grab handles 24 are provided at each comer which enable the user to alter the 

10 height and width as necessary. Finally, marker 25 is provided that indicates the centre 
of the short axis of the text selection window 22; marker 26 is provided that indicates 
the centre of the long axis of the text selection window 22. The user can drag markers 
25 and 26 to the appropriate position using either the conventional 4-way movement 
controller 23 or, where the computer is pen driven, by dragging marker 25 or 26 with 

15 the pen. 

Text selection window 22 is divided into four equal width sections 27 A, 27B, 27C and 
27D, The bit maps in the segment of the facsimile image 21 which underlie each equal 
width section are processed by being subject to a simple magnification process which 

20 enlarges and maps the contents of each section 27A to 27D into a respective section of 
the viewing window 30; viewing window 30 is divided into four rows, 30A, 30B, 30C 
and 30D, Section 27 A of the facsimile image 21 is mapped into row 30A; section 27B 
of the facsimile image 21 is mapped into row 30B; section 27C of the facsimile image 
21 is mapped into row 30C; section 27D of the facsimile image 21 is mapped into row 

25 30D. In this way, an entire line of bit mapped text can be easily read in viewing 
window 30. To view the next line, the user simply selects the on-screen "return" key 
31 if the computer is pen driven. Text selection window then automatically moves 
down the correct distance to overlie the next line of bit-mapped text. In some kinds of 
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computer, for instance small personal organisers with integral keyboards, the actual 
"return" key is hit to move the text selection window to the next line down. In each 
case, the pattern recognition engine may be used to ensure that the window 22 is 
positioned at the optimum position to enable the next line of text to be read. Another 
5 approach to viewing the next line down is to simply manually re-position text selection 
window 22 using controller 23 or marker 26. 

A further feature is that the viewing window can be used not only merely to display 
single lines of the original facsimile image 21 broken down into a number of rows as 

10 described above; in addition, it can be used to select an area of the image 21 to be 
viewed without being split into segments and then re-assembled. Hence, by selecting 
the magnifier option button 32, the viewing window 30 becomes a conventional 
"magnifying glass". In addition, the text selection window is shrunk down to a size 
which enables the whole of the image that it underlies to be displayed within the 

15 viewing window 30. This feature enables the user to switch quickly from the 
"Fax View" embodiment of this invention to a conventional "magnifying glass". 

It is preferable if words in the facsimile image 21 are not chopped into two; this would 
happen if the boundary between adjacent sections 27 of text selection window 22 

20 intersects words. The present embodiment addresses this problem by dynamically 
altering the width of each section 27 A, 27B, 27C and 27D so that no bit-mapped text is 
intersected in this way. A pattem recognition engine searches a narrow strip on the left 
hand border of each section 27 A, 27B, 27C and 27D for the presence of an underlying 
bit mapped text. If underlying text is indicated, the width of the relevant section is 

25 incrementally varied and the search of the narrow band repeated. The process continues 
until it is detemndned that no text underlies the narrow band. The process is then 
repeated for the next section along until all sections across the entire width of the line 
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have been sized to ensure that no underlying word will be split into two when viewed 
in the viewing window 30. 



5 



10 
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1 . A method of displaying an image of a document on a display screen comprising 
5 the steps of: 

[ 1 ] receiving, from a remote transmitting device, a bit-mapped image of a 

document comprising text; 
[2] dividing a whole or part of a line of text of the document into segments; 
[3] processing the segments so that text in each segment can be displayed at 
10 a legible size and re-arranging segments which are positioned adjacently 

in a line or a part of a line into two or more rows and; 
[4] displaying the re-arranged segments on the display screen. 



15 2. The method of Claim 1 wherein each segment is displayed within a viewing 
window divided into rows, with the first row of the viewing window containing 
processed text from the segment from the far left of the line, the next row down 
containing the processed text from the next segment along and subsequent rows down 
contain the processed text from adjacent segments along the line. 

20 

3. The method of Claim 2 wherein the user can select the text which is to be 
processed from the document image. 

4. The method of Claim 1 wherein the following step occurs after step [2]: 
25 displaying either some or all of the image either (i) at a magnification at which it is not 

readily legible or (ii) as a picture representative of the image. 



BNSDOCID: <WO 982291 0A1_I_> 



wo 98/22910 



PCT/GB97/03126 



12 

5, The method of Claim 4 wherein selecting the text is achieved using a movable 
selection window or cursor which can be moved over portions of the illegible image or 
the picture to the desired line to be viewed in the viewing window, 

5 6. The method of Claim 5 wherein the height and/or width of the movable 
selection window can be varied. 

7, The method of Claim 6 wherein the output of a pattern recognition engine is 
operable to control the height of the movable selection window so that it approximates 

10 totheheightof alineof text. 

8, The method of Claim 7 wherein the pattern recognition engine looks at a pre- 
determined narrow region in the movable selection window and the height of that 
window is varied in incremental steps until an output is generated indicative that the 

15 narrow region does not overlie any text* 

9, The method of Claim 7 wherein the pattern recognition engine looks at a pre- 
determined narrow region in each segment of the movable selection window and the 
width of that segment is varied in incremental steps until an output is generated 

20 indicative that the narrow region does not overlie any text 

10, The method of Claim 5 wherein the movable selection window can also be used 
to select a portion of the image to be magnified and displayed in the viewing window 
without being divided into segments and then re-assembled in a different arrangement 

25 

1 1 , An apparatus for displaying an image of a document on a screen comprising: 
[1] a receiver for receiving, from a remote transmitting device, a bit-mapped 

an image of a document comprising text; 
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[2] a processor programmed (i) to process the document by dividing a 
whole or part of a line of text into segments; (ii) to process the segments so that their 
contents are of a legible size and (iii) to re-arrange the segments which are positioned 
adjacently in a line or a part of a line into two or more rows so that their contents can be 
easily read and 

[3] a display to display the re-arranged segments. 
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